8/01/06 



National Association 
of State Universities 
and Land-Grant Colleges 




Toward a Voluntary System of Accountability 
Program (VS A) For Public Universities and 

Colleges 



Peter McPherson, President 
and 

David Shulenburger, Vice President for Academic Affairs 



National Association of State Universities and Land-Grant Colleges 
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F 0 t P II J 1 1C U N IP £ l S IT IE S AND COLLEGES, explored the nuances of public university 
accountability. They have enjoyed wide exposure and stimulated conversation, 
particularly among NASULGC and AASCU Presidents, Chancellors and Provosts. This 
paper comes out of these conversations and further develops the elements that appear to 
be important in creating the Voluntary System of Accountability Program (VSA). 



INTRODUCTION 



Higher education historically has played a vital role in this country, enabling 
social and economic mobility for individuals and creating cultural, scientific and 
technological progress for society as a whole. NASULGC and AASCU have embarked 
on an effort to enhance university accountability and our members broadly recognize the 
gravity of this mission. “We have the keys to the future and the obligation that comes 
with holding them,” said a provost at a meeting discussing accountability. In an era in 
which Thomas Friedman’s book, The World is Flat, has served to raise awareness and to 
catalyze concern about the competitiveness of the American economy and, consequently, 
the future welfare of every American, holding those keys confers an even greater 
obligation. Our universities, particularly public universities, are obligated to carefully 
focus their resources upon their individual segments of their missions and to effectively 
and efficiently utilize them. 



It is against this background that the university community understandably has 
been called upon to be even more accountable. Every public university is engaged now 
in serious and ongoing accountability appraisal with significant time and resources 
dedicated to the task. The academy’s commitment to accountability is real. 
Nevertheless, we are prepared to supply more and better accountability information to 
our diverse stakeholders. 

Accordingly, we suggest what we believe will be a reasonable and helpful set of 
undergraduate education accountability measures for universities that we label the 
“Public Universities and Colleges Voluntary System of Accountability for Undergraduate 
Education” (VSA). The set contains key measures that provide basic data to help 
students (and parents) find the university that best fits their needs, gauges the degree to 
which students on various campuses engage with the learning process and helps 
stakeholders assess the increase in undergraduate student learning that is occurring at 
universities. Transparency in collecting and presenting this information is important if we 
hope to have a voluntary accountability system acceptable to the public. 

Focusing on accountability for undergraduate education, one of several important 
functions of public universities satisfies a need, but it should not be construed as 
accounting for the totality of university output. The Council on Competitiveness 
recognizes the importance of undergraduate education, especially in the science, 
technology, engineering and mathematics (STEM) areas, as crucial to the nation’s future. 
However the Council also recognizes that university graduate education and research are 
similarly critical. Likewise, the Kellogg Commission on the Future of State Colleges and 
Land-Grant Universities described the engagement universities provide as vital to 
domestic communities and the world. These important products of universities are 
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evaluated extensively and we share those evaluations freely. Universities should and will 
continue such reporting. 

We suggest a voluntary accountability system in which universities may elect to 
participate. This paper represents the beginning of an effort and there clearly must be a 
community process developed to monitor and make appropriate modifications as time 
goes on. If the academy can agree on this meaningful but manageable set of 
accountability measures, we would generally satisfy the public by providing more useful 
information. There would be economy and value in that result. Moreover the academy 
would retain the ability to choose measures that we can use to improve our ability to 
educate our undergraduate students. 

We urge that this accountability data be used only to compare specific universities 
with their own past performance and with the performance of comparable universities. 
We think that regional accreditors will want to consider using the set of accountability 
measures to help satisfy their standards and governing boards may also want to substitute 
these measures for the information they currently require of institutions. 

We begin with the set of first principles that presidents and provosts felt to be 
important tests of the viability of a voluntary system of national public university 
accountability. 



3 




FIRST PRINCIPLES ABOUT ACCOUNTABILITY 
MEASUREMENT DERIVED FROM THE DISCUSSIONS 



1- As educational institutions, our first obligation is to develop and use those 
accountability measures that provide the maximum assistance in improving the 
educational outcomes of our students. 

2- The interests of higher education are best served if our accountability measures 
focus on the increase in educational value for which we are responsible and not on 
the initial intellectual and educational endowments our students possess at 
matriculation. 

3- All of our accountability measures should be transparent. Both the methods by 
which they are derived and the results of those measures should be publicly 
disclosed. 

4- The immense diversity of American higher education institutions is a valuable 
attribute. For that reason universities should be compared against their own past 
performance and with other universities with similar missions, academic programs 
and admissions practices. 

5- Even comparisons of similar universities should be limited to individual 
accountability measures, not indices composed of multiple accountability 
measures. 

6- Because our resources are used best when directed toward our educational, 
research and service missions, only those resources absolutely required should be 
allocated to this enhanced accountability effort. Consistent with this aim: 

• The rigorous set of accountability measures proposed here should be 
evaluated by accreditors and governing boards and, where possible in 
order to help contain cost, substituted for the accountability measures 
currently mandated or generally used to provide evidence that standards 
have been met. 

• Where appropriate, sampling should be employed in measuring 
outcomes and campus learning climate rather than measuring every 
student at significant costs. 

7- The national interest requires developing measures to assess broad international 
trends related to student learning; access and student progress in higher education; 
public investment in research, education, student support and other valued 
purposes in higher education; and public return and national economic 
competitiveness on investments in higher education. 





CONCLUSIONS ABOUT CONSTITUENCY ACCOUNTABILITY 



There has been widespread agreement among presidents and provosts who have read 
versions of this paper that we should be and/or are expected to be accountable to three 
constituencies: 

1- Prospective students, current students and their parents; 

2- Faculty and campus support staff; 

3- Public policy-makers and public and private funders of higher education. 

1-Prospective students, current students and their parents 

The impact of undergraduate education lasts a lifetime as it lays the foundation for 
graduate study and one’s future occupation. In choosing a university for undergraduate 
study, students and their parents have made a decision that will have fundamental effects. 
We owe them information that would be helpful in making this choice. When one 
reflects on the questions students and parents ask, what is meant by “Student and Parent” 
information becomes reasonably clear. 

• How much will it cost to go to your school? 

• Are there costs beyond the required base tuition? What are they? 

• What kind of jobs do your graduates get? 

• How much does it cost to live in the residence hall? 

• May I live off campus during my freshman year? 

• Will the college-level coursework that 1 have taken in high school or at a 

community college transfer? Will all of it transfer or just some of it? 

• How long will it really take me to earn a degree? 

• What is your graduation rate ? 

• How many students fail to receive a degree? 

• How likely are students like me to graduate from your university? 

• What chance of getting a degree do transfer students have? 

• Are student jobs available on campus? 

These questions and many others are regularly directed to admissions personnel, 
presidents, provosts, other faculty, and even current students. 

In a world in which financial aid programs struggle to keep up with demand, and 
the burden of paying for undergraduate education increasingly falls on students and/or 
their parents, we should provide as clear and concise answers to their questions as we 
can. These answers should allow them to compare institutions by the measures most 
relevant to them. The ultimate result of providing such information should be to improve 
the fit between student and university, resulting in greater satisfaction and improved 
educational success. Improving the initial fit for students should reduce transfers among 
four-year colleges and ultimately reduce the cost of education to the student. 
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2- Faculty and campus support staff 

Information on student learning at one’s own university and at comparable 
universities provides helpful data to faculty and staff. Derek Bok in his recent book, Our 
Underachieving Universities , urges all universities to measure educational outcomes so 
that we can know which educational practices are effective. Core educational outcomes 
measurement will provide data to help faculty choose among teaching strategies based on 
their relative success. Accessing key measures of educational effectiveness at other 
institutions will be valuable in determining if curricular innovations, methodologies and 
expectation levels present in those universities better achieve desired results than do those 
in one’s own university. Clearly, providing this kind of evaluative and comparative 
information to faculty and staff is essential to improving the university. 

Because improving student learning is our primary goal, universities must be 
accountable to those who teach and interact most with our students. Faculty and staff 
need feedback about student engagement and their success in educating students. The 
kinds of data from the National Survey of Student Engagement (NSSE) or the 
Cooperative Institutional Research Program (CIRP), which we label “Student 
Engagement with Campus Data,” help faculty and staff to know whether their students 
are being expected to read, write and participate in class discussions as much as at 
comparable universities. The data tell them whether their university’s students are 
reasonably engaged with their institution. This information helps benchmark campus and 
faculty practices against those in comparable universities. In this more controlled 
environment, change can be introduced and the impact of that change on student 
engagement with the campus and learning can be thoroughly evaluated 

3- Public policy-makers, members of the public and public and private funders of 
higher education 

Public policy-makers, members of the public and public and private funders can choose 
to bestow their attention and funding on higher education or myriad entities that compete 
for funding with universities. Clearly included in this grouping are legislators and 
members of boards of trustees but also foundation board members and, equally 
importantly, alumni. Each wants to know the conditions under which the public is 
provided access to higher education, who is receiving the benefits and how much learning 
is occurring. We believe that the data elements in VSA will be of value to the diverse 
members of this group. 



COMPONENTS OF ACCOUNTABILITY AND THEIR ATTRIBUTES 

What is suggested by VSA is essentially a bundle of data focused on 
undergraduate education whose individual elements deal with important and legitimate 
concerns. The hope is that those that use the elements will be able to gain a fair and 
useful view of important facets of each participating university. Each of the groups 
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described above should benefit from improved systems of accountability. We suggest that 
a packet of information could provide that accountability if it included: 1) student and 
parent information, 2) student campus engagement data of the sort that can be derived 
from NSSE or CIRP and 3) value-added core educational outcomes information that 
could be derived from national instruments such as the Collegiate Learning Assessment 
(CLA), Measure of Academic Proficiency (MAP), Collegiate Assessment of Academic 
Proficiency (CAAP) or Graduate Record Examination (GRE). 

Comparable Data. For the data set to serve the needs of students in their search for the 
right university and to satisfy public accountability demands, that packet must contain 
comparable data across universities. This data set by design must incorporate 
measurement of items common to all universities offering baccalaureate degrees. 
(Universities, of course, will continue to have more thorough data sets available for other 
management and evaluation purposes.) The student and parent information component of 
VS A will answer questions and supply data about elements of interest to those selecting a 
university in a straightforward manner because the conventions used to define and report 
the data will be common across all participating universities. The student campus 
engagement data has been defined by the two existing instruments that collect it and can 
therefore be uniform across campuses that choose to use either of them. Finally, despite 
the significant and admirable diversity of our universities, the resulting core outcomes 
data included will be restricted to a small set of tests that reflect, among other educational 
outcomes, the commonly expected core educational outcomes of undergraduate 
education: development of critical thinking, analytical reasoning and written 

communication skills. 

Disclosure . Both the student and parent information and the core educational outcomes 
data must be made available publicly if they are to satisfy student and public 
accountability needs. Since the use of core educational outcomes tests is new to most 
universities, they will understandably want to have a period in which they find the best 
methods of administration and use the results to adjust their educational programs before 
making the results of the outcomes tests public. We recommend that this period of 
experimentation be three or four years. Similarly, most universities now view the results 
of NSSE/CIRP as diagnostic in nature and choose to use them to help guide the internal 
agenda for change rather than have them as a public indicator of their performance. But 
the public and prospective students have an interest in the summary reports of 
engagement data and in most states have a right to see it under the various state freedom 
of information acts. For those reasons we recommend that summary engagement data be 
part of the VS A data set while recognizing that many universities, especially those that 
have previously not had the benefit of information from such instruments, will want to 
have an opportunity to adjust their practices and operations once they administer the 
instruments. In addition, universities commonly choose to administer these instruments 
only every three years. Accordingly, we recommend that data like this not be made 
available as part of the VS A during the first four years of the operation of the 
accountability system. Thus, after four years the entire VS A data should be publicly 
available from each participating university. 
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VS A will not serve all needs. Our intention is that it serve the set of needs 
common to those seeking, delivering, overseeing, developing polices for or funding 
undergraduate degrees at U.S. universities. We fully recognize that there is more 
specialized data that individual schools must collect for their own use. Indeed, for 
purposes of measuring the effectiveness of individual majors, universities will need to 
continue a regular set of program reviews. 

Only appropriate comparisons should be made. Universities participating in VS A 
will use the same format and terminology for the set of core accountability measures and 
the public reporting of them. As a result comparison of universities on specific 
dimensions of interest to potential students will become easier. We must caution that 
there is no intelligent way to generally compare institutions that are essentially different 
in focus. 

Only comparison among comparable universities is appropriate. The best 
comparison is of one university with itself over time. In order to enable comparison other 
than self-comparison, it is necessary to group universities appropriately. Frankly, we at 
NASULGC and AASCU cannot and should not establish groups for the sake of 
comparison. Individual universities generally have defined peer groups. We encourage 
universities to publicize the names of universities within those groups so that the 
comparisons can be made with defined peers. A limited form of comparability exists 
among institutions with similar admissions criteria so we recommend that the data set 
include figures on both entering average SAT and ACT scores and entering average 
GPAs. We must realize, however, that even similar admissions standards universities 
may differ markedly on funding, curricular approaches, resident or commuting 
populations, etc., and may not be fundamentally comparable at all. 

We note that the Education Trust website, College Results Online, utilizes an 
algorithm to develop university peer groups and also permits the user to specify alternate 
peer groups. None of the above is meant as a criticism of their approach or of those who 
use other algorithms for this purpose. The user of any peer group should always ask 
whether any grouping specified is really made up of “peers” for the purposes for which 
they wish to use the grouping. 

We vigorously oppose creating any overall ranking scheme based on the bundle 
of accountability measures we recommend here. Of course, we understand that public 
data is just that and will be used by others. We advocate that common accountability data 
be presented by institutions with the user of the data encouraged to place whatever weight 
on the individual data elements she/he prefers. Given that missions, student selectivity, 
urban/rural location, student body composition, etc., differ by university, combining the 
accountability data into any form of index for the purpose of ranking these universities 
would represent a disservice to them. Such indices may also constitute a disservice to our 
three core constituencies as the individual data elements are tailored to answer specific 
questions for specific groups. Combining data into any sort of index will serve to 
obfuscate what the carefully defined, distinct data elements are intended to clarify. 
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A Brief Focus on the Set of Measures 

Selection of Specific Accountability Measures and System Implementation 



Through the distribution of this paper and subsequent conversations and campus 
visits with presidents and provosts we are seeking feedback to further refine the concepts 
in the VS A. When the core concepts of VS A are widely accepted in the public 
university community, we envision developing a process by which the universities 
inclined to participate in VS A would work through the national associations representing 
them (or through some other acceptable organization) to decide upon the specific 
elements of student and parent information, the specific set of student engagement with 
campus measures and the specific set of core educational outcomes measures to be 
utilized. Once that bundle of measures constituting the VS A is agreed upon, individual 
universities would then voluntarily choose whether to become a VSA university. 

VS A universities would maintain their VSA information on their own publicly 
accessible web sites. Since the data would be in a common format and the definitions 
would be uniform, those wanting to access any university’s information could do so 
through the web. There would be no need for any agent of the academy to hold the data 
for all universities in a common repository. Third parties might choose to harvest the 
data and build repositories from it as they do with other organizational data such as 
USDE’s IPEDS that is publicly maintained. 

Because all of these processes would take some time to develop and implement, 
and because universities need time to adjust practices to the findings from the initial 
measurements, we do not envision the first complete reports from individual universities 
subscribing to the national standards of accountability to be available to the public until at 
least four years after agreement is reached to proceed with this notion. Thus complete 
public reports would not be available until fall, 2011, but reports on the student and 
parent information would be available earlier. 
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Three Components of Public University Accountability 

As described above there was interest/agreement that three specific components 
should be included in VSA: 

1. Student and Parent Information 

2. Student Campus Engagement 

3. Core Educational Outcomes 



1. Student and Parent Information. This accountability category primarily focuses on 
students and their parents. Accordingly, this data should be chosen after first asking the 
question: “ What information would be most helpful to prospective students and their 
parents in deciding which university best fits their educational wants and needs?" 
Clearly various actual price of attendance figures, degree offerings, living arrangements, 
graduate placement statistics, graduation rates, transfer rates, employment facts (concerns 
about the suitability of available data sources to provide these facts have been detailed in 
the appendix to the earlier papers), etc., would be included here. To facilitate appropriate 
comparison, the data sets will include the list of universities considered by each 
participating university to be its peers in undergraduate education and also will include 
entering test scores such as average ACT/SAT and entering GPA. We do not envision 
the data set incorporating all the data elements universities now must report to the federal 
government. In order to maximize the usefulness of the accountability system data, the 
system may include some of that data. 

Any valid accountability data set must include student success measures such as 
graduation rates. Unfortunately, a true picture of graduation rates cannot be generated 
from data currently available, as information on where students transfer within the higher 
education system cannot be reliably ascertained from existing data sets. The academy is 
short-changed by not having robust data available to develop the complete picture. 
Therefore we support serious conversations among higher education, congressional and 
U.S. Department of Education leaders leading to the development of a data set that will 
overcome the current limitations on measuring graduation and transfer rates. Any 
resulting data set must protect the privacy of individual students and be utilized only for 
legitimate higher education accountability purposes. 

2. Student Engagement with the Campus. Both the NSSE and the CIRP are proven 
programs that enable campuses to compare student self-reports about learning-related 
behaviors. This data has proven valuable to faculty and staff by enabling them to make 
alterations in both campus and classroom environments to provide greater stimulation to 
students. One of these two measures should be part of the accountability package. The 
output of each of these surveys is complex and lengthy. While it may be of limited 
interest to some outside the university, its primary value is to faculty and staff as they 
seek to understand the learning environment and improve it. Public reporting through 
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VS A should include only summary student engagement data, not the very detailed 
components from which the summary elements are developed. The value of NSSE and 
CIRP instruments is described in some detail in our IMPROVING STUDENT LEARNING 
paper. 



3. Core Educational Outcomes. Each of the groups to whom we need to be accountable 
has asked for data, sometimes often, on the development of critical thinking, analytical 
reasoning, and written communication skills. Core educational outcomes like these are a 
primary component of undergraduate education and should be measured for the 
accountability purposes under discussion here. Tests l ik e the CLA, MAP and CAAP 
measure them to varying degrees, even though the latter two tests were designed to 
measure general education, not core education, outcomes. Even the GRE, although it 
measures abilities and learning at a much higher level than the other three tests, also 
reflects accomplishment of the core outcomes. While each university may aspire to 
develop outcomes in addition to these three sets of skills, these three appear to be 
common outcomes undergraduate education is expected to produce at most 
NASULGC/AASCU institutions. Research on the ability of standardized tests to 
measure these core education outcomes is ongoing and must continue until sufficient 
numbers of students at institutions of varying sizes and types have taken the exams. Only 
then can we reach firm conclusions on the adequacy of the various tests to measure these 
core education outcomes. 

What we do know about performance on standardized tests like the CLA, MAP, 
CAAP and the GRE is that it reflects both the initial ability of the student entering the 
institution and the educational value added by the institution. For institutional 
accountability purposes, only a measure of the latter is desired. One can directly adjust 
the performance of seniors on core educational outcomes exams for entering ability (as 
measured by ACT or SAT scores when the students were accepted for admission to the 
university) or one can administer the core educational outcome test twice 
(freshman/senior years) and use the difference in test scores to judge how much core 
educational value is added by the institution. Of course many things happen to a student 
between the freshman and senior years that are unrelated to the education provided by the 
university, e.g., travel, development of a wider social network, summer and academic 
year jobs, and each of these may have an effect on standardized test scores. Nonetheless, 
it is clear that selecting one of the normalization techniques is required to refine the 
measurement such that it comes closer to approximating only the value added by the 
university. 

We are aware that controversy surrounds value-added measurements. Both 
freshmen and senior CLA test results are highly correlated with incoming ACT/SAT 
results. (The same close relationship would probably be found with ACT/SAT scores if 
CAAP, MAP and GRE were also given to freshmen and seniors.) The 2005-06 CLA 
round of campus administrations however, found significant, positive value added. 
Measurement difficulties do not diminish our resolve that value added is the appropriate 
outcome measure upon which to focus. It does mean that the developing science of 
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value-added learning measurement must be sensitive to these relationships and that a 
value-added measure initially chosen by a university may have to be reconsidered as 
additional research results are amassed. 

One therefore must regard value-added measurement as still in the “experimental'’ 
stage. Accordingly, we cannot at this time recommend the selection of a single test for 
all universities subscribing to a national public university accountability system. Instead, 
we recommend that a set of three or at most four outcomes tests be selected by the 
universities participating in VS A and that each university in the interim select the one 
test from that set that measures best the core educational outcomes goals that the school 
has designed its curriculum to produce. After the three or four years, schools would 
consider the ability of its chosen test to measure core educational outcomes and confer 
with other universities about the performance of the tests they chose to use. Perhaps at 
that time a single “best” core outcomes test will emerge and a single test could be 
adopted across all VS A universities. Perhaps individual universities will find that 
different specific tests fit them best and no single outcomes test will emerge. If this 
eventuality arises it is conceivable that research could produce a set of “cross walks” that 
would permit one to convert the scores and sub-scores on any test into the scores and sub- 
scores on the other tests. (Note that such a cross walk is in common use to convert 
ACT/SAT scores with the result that many schools are willing to accept either score for 
admissions purposes.) 

We are aware that having participating universities measure core educational 
outcomes, at least initially, with one of three or four different tests does not immediately 
fully satisfy the goal of producing “comparable” value-added outcomes data although it 
may do so by the end of the experimental period. The newness of one of the tests (the 
CLA), the likely emergence in the market of other tests and the lack of familiarity of 
some universities with any of the existing tests, suggests that the route described above is 
the most prudent and educationally sound. 

While value-added measurement is for most purposes the appropriate measure for 
public universities, the educational outcomes test score itself has importance for some 
universities. Interpreting the meaning of specific test score performance levels may be 
problematic. Any value-added approach involves generating raw test scores, so the 
value-added focus we recommend does not preclude the availability of test score data. 

A serious problem that remains for general education assessment testing is the 
difficulty of ensuring that the students tested are motivated to perform at their best level 
while taking the tests. This problem is generally not a matter of concern when students 
are taking classroom exams or the ACT/SAT or GRE exams as the test-takers are 
motivated by self-interest to do well. General education standardized exam performance, 
by contrast, has no impact on the test taker but may have consequences for the university. 
In a sampling scheme like that employed by the CLA, having only a small percentage of 
the typical sample of 100 students treat the testing experience as a lark or malevolently 
decide to do harm to their institution, can seriously bias the general education outcome 
the campus reports. Various methods have been suggested to correct this problem, 
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ranging from embedding the test into a capstone course with performance on the exam 
counting as part of the course grade, to awards of iPods to top performers on the exam. 
To avoid this motivation bias problem some campuses may even decide that they will not 
use a sampling strategy in measuring general education gains but will instead administer 
CLA-like measures to every student in a required capstone course. There are many 
techniques available to avoid such bias and we do not endorse any one technique here. It 
is clear that every university will need to tailor a method to its peculiar requirement to 
ensure that motivation problems do not bias test results. 

Similarly, where sampling is utilized, uniform sample selection practices must be 
implemented to ensure that the tests are administered to a truly random sample of 
students. Every campus naturally wants to show its best face and thus there will be the 
temptation to administer the test to non-random samples of students, e.g., to high-ability 
individuals. Unless uniform sample selection procedures are agreed to and rigorously 
observed on every campus, the willingness of campuses to participate in any general 
education assessment venture and particularly their willingness to make results public 
will be undermined. 

Unfortunately, there are no standardized tests that measure campus-wide the value 
added for the entire undergraduate educational experience. Constructing such a test is 
extraordinarily difficult as campuses have diverse sets of majors and degree programs 
and the likelihood of getting agreement on common educational outcomes is low. In 
addition, the sheer number of majors at U.S. universities is in the hundreds so the effort 
to develop a comprehensive suite of major- specific outcomes tests is mammoth. There 
are, however, outcome measures that indicate overall campus educational outcomes for a 
handful of specific purposes. They include the LSAT, MCAT, GMAT and the GRE 
specialized disciplinary exams. We are not now recommending using these measures as 
part of this educational outcome suite because they measure the experience of only very 
small and highly self-selective samples of the student population at some universities. 
We also do not know how these measures relate to core educational outcome measures. 
Further research could help conclude whether including these measures in the bundle 
would be justified. 

Measurement Frequency 

Data on each of the measures need not be collected annually. Many users of 
NSSE administer it every two or three years as the campus climate seldom changes 
suddenly and schools wish to allow time for interventions they implement to have an 
opportunity to work prior to re-administering the test. Similar considerations may lead to 
the decision to administer other measures on a biennial or triennial schedule. Explicit 
decisions about the frequency of testing must be made so that the expense of 
administering the instruments or collecting the data is commensurate with the value of 
the data obtained. 
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IMPLEMENTATION OF VSA 

Representatives of universities agreeing in principle to establish the VSA would be 
convened to decide upon the data elements/measurements that would constitute each 
portion of the data set and the conventions that would be observed in data definition, 
measurement and presentation. 

Universities that agree to subscribe to the system as defined by the above process and 
who agree to observe its conventions will be known as VSA. The minimum 
components of public accountability that would be included are: 

1. The full set of student and parent information measures agreed 
upon; 

2. Student engagement with campus measures-NSSE or CIRP or other 
measure selected; and 

3. Core learning outcomes measures: Use of one of the three or four 
outcomes tests (CLA, MAP, CAAP, and GRE) agreed upon for the 
experimental period. 

Thus, every university that chooses to be a member of the Voluntary System of 
Accountability would utilize and report to the public precisely the same student and 
parent information measures, one of the three or four agreed upon learning outcomes 
measures and summary results of a student engagement with campus survey. 

The system, once in place must not be static as the educational and technological 
environment changes rapidly. Thus, a mechanism to review the components of VSA 
and change them as needed must be put in place. 



FINAL THOUGHTS 

The Role of Regional Accreditors. Regional accreditation associations serve a quality 
assurance role of major importance to American higher education and the public. We do 
not wish to replace this role with a national set of public university accountability 
standards. We ask that the six regional associations consider declaring that they will 
accept the resulting set of accountability measures as appropriate evidence for some 
standards. Doing so enhances the goal of achieving transparency and comparability on a 
national level. We understand that most regional accreditation standards already permit 
the university seeking accreditation or reaccreditation to choose a variety of ways to 
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satisfy their standards and that this request is therefore minimal. In order to further 
promote adoption of these accountability standards, the regional accreditation 
associations should be convened at an appropriate time to consider how to achieve this 
objective. 

For the sake of Parsimony. If universities agree on a set of national undergraduate 
accountability standards they are agreeing to gather and supply a considerable volume of 
detailed information on a regular basis at significant expense. University budgets are not 
generous and all would rather expend as many scarce resources as possible directly on the 
educational mission. If such voluntary measures are put in place we believe the 
universities using them will have a strong basis for asking state and federal government 
agencies, governing boards, regional accrediting agencies and others to agree to a 
moratorium on requests for more accountability measurement and to begin a review of 
existing requirements with a view to determining whether some might be eliminated. If 
an accountability system provides a set of consistent, clear and comparable data and 
satisfies the various stakeholders, some other measures could be withdrawn. It is our 
hope that the measures chosen would replace most existing measures, not add to them. 

The Need for a Continuing Process. Agreeing to create and implement a voluntary 
national set of accountability standards cannot be the end of this process. Since the core 
outcomes measure must be subjected to a period during which experimental use of the 
various available tests occurs, a mechanism to evaluate the status of the candidate tests 
must be put in place. In addition, what is an adequate accountability process today may 
not be considered adequate tomorrow due to changes in the educational environment and 
technology. Thus a mechanism must be created to provide oversight to the agreed upon 
set of standards and to permit the system to be responsive to changes in the environment 
and lessons learned. 



CONCLUSION 

The set of Public Universities and Colleges Voluntary System of Accountability 
for Undergraduate Education standards described here amounts to self-evaluation by 
those universities that choose to subscribe to it. This self-evaluation system arises from 
the same tradition as does our voluntary accreditation system that has served both the 
public and the academy well. It preserves the diversity of the U.S. higher education 
system while permitting the public to judge the various facets of individual universities. 

While these suggestions are based on feedback from a substantial number of 
correspondents who read the predecessor papers, IMPROVING STUDENT LEARNING IN 
HIGHER EDUCATION THROUGH BETTER ACCOUNTABILITY AND ASSESSMENT and ELEMENTS 
OF ACCOUNTABILITY FOR PUBLIC UNIVERSITIES AND COLLEGES, we hold no illusion that it 
cannot be improved. We ask for your comments and suggestions. Please send them to 
dshulenburger@nasulgc.org or pmcpherson@nasulgc.org . 
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